Construction layout using augmented reality

ABSTRACT

An augmented-reality system is combined with a surveying system to make measurement and/or layout at a construction site more efficient. A reflector can be mounted to a wearable device having an augmented-reality system. A total station can be used to track a reflector, and truth can be transferred to the wearable device while an obstruction is between the total station and the reflector. Further, a target can be used to orient a local map of a wearable device to an environment based on a distance between the target and the wearable device.

CROSS-REFERENCES TO RELATED APPLICATIONS

The following three U.S. patent applications (including this one) arebeing filed concurrently, and the entire disclosure of the otherapplications are incorporated by reference into this application for allpurposes:

-   -   application Ser. No. 16/924,648 filed Jul. 9, 2020, entitled        “Augmented Reality Technology as a Controller for a Total        Station”;    -   application Ser. No. 16/924,650 filed Jul. 9, 2020, entitled        “Layout Workflow with Augmented Reality and Optical Prism”; and    -   application Ser. No. 16/924,651 filed Jul. 9, 2020, entitled        “Construction Layout Using Augmented Reality.”

BACKGROUND

This disclosure relates in general to surveying systems and augmentedreality. Surveying systems determine positions of points relative toeach other and/or to the Earth. Surveying can be used in manyapplications by land surveyors, construction professionals, and civilengineers. Surveying often uses specialized and/or expensive equipment,such as laser levels, surveying rods, total stations, laser scanners,and GNSS (Global Navigation Satellite System) receivers.

In augmented reality, one or more virtual objects (e.g.,computer-generated graphics) can be presented to a user in relation toreal-world objects. Augmented reality can include a see-through displaywith a virtual object shown to a user on the see-through display. Anexample of an augmented-reality system is the Microsoft HoloLens.Another type of augmented reality is overlaying a virtual object on animage of the real world. For example, a smartphone camera is used toacquire an image of objects in the real world. The smart phone thenoverlays a graphic on the image of the real world while presenting theimage on a screen of the smart phone. Artificial reality is sometimesused to refer to both augmented reality and virtual reality.

BRIEF SUMMARY

This disclosure relates to surveying systems. More specifically, andwithout limitation, this disclosure relates to using artificial reality(e.g., a Microsoft HoloLens) in combination with one or more surveyingsystem (e.g., a robotic total station) for more efficient measurementand/or layout.

In certain embodiments, a system for identifying points of interestduring construction layout comprises: a base station configured tomeasure angles and distances from the base station to a reflector togenerate position data of the reflector in relation to an environment; awearable device; and one or more processors. The wearable devicecomprises: a hardhat; the reflector, wherein the reflector is fixedlycoupled with the hardhat; an optical display fixedly coupled with thehardhat; a camera fixedly coupled with the hardhat; and/or a graphicsengine configured to provide a local map based on images acquired by thecamera. The one or more processors are configured to: receive theposition data; orient the local map of the wearable device to the basestation using the position data; and/or present to a user of thewearable device, using the optical display, a virtual object in relationto the environment, based on orienting the local map of the wearabledevice with the base station. In some embodiments, the optical displayallows real-world light to pass through the optical display; thereflector is a prism; the reflector is a pattern of retroreflectivestickers; the base station comprises a theodolite to measure anglesbetween the base station and the reflector; the position data includeslocation data and orientation data; the position data includes locationdata, and orientation data is obtained measurement of an edge obtainedfrom the local map; the reflector is a first reflector and the hardhatcomprises a second reflector in a known orientation to the firstreflector; the base station comprises an electronic distance measurement(EDM) device for measuring a distance from the base station to thereflector; and/or the system further comprises a tracking camera,separate from the base station and the wearable device, wherein thetracking camera provides imagery data used to determine position of thewearable device with the environment. In certain embodiments, anapparatus comprises a wearable device and one or more processors. Thewearable device comprises a frame; a marker, wherein the marker isfixedly coupled with the frame; an optical display fixedly coupled withthe frame; a camera fixedly coupled with the frame; and/or a graphicsengine configured to provide a local map based on images acquired by thecamera. The one or more processors are configured to receive positiondata relative to an environment; orient the local map of the wearabledevice to the environment using the position data; and/or present, usingthe optical display, a virtual object in relation to the environment.The marker can include a retroreflector, a two-dimensional design, amachine readable code; and/or a one-dimensional or two-dimensionalbarcode. The apparatus can comprise a camera on a back of the frame usedto track a location and/or an orientation of a base station with respectto the frame. In some embodiments, a base station comprises a camera;the camera is separate from the base station; and/or the camera isconfigured to acquire images of the marker used to determine relativeposition of the marker in relation to the base station.

In certain embodiments, a method for using augmented reality inconstruction layout, the method comprises: measuring a position of areflector in relation to a base station to obtain position data, whereinthe reflector is fixedly coupled with a hardhat, and the base station isconfigured to measure angels and distances from the base station to thereflector in relation to an environment; generating a local map based onimages acquired by a camera that is fixedly coupled with the hardhat;orienting the local map to the base station using the position data;and/or presenting to a user, using an optical display fixedly coupledwith the hardhat, a virtual object in relation to the environment, basedon orienting the local map with the base station. In some embodiments,the method further comprises orienting a model to the environment, andwherein the virtual object is from a feature of the model; and/ororienting the base station to the environment.

In certain embodiments, a system for using augmented reality forbuilding layout comprises a surveying pole; a base station; a wearabledevice comprising a camera; and/or one or more memory device containinginstructions that, when executed, cause one or more processors toperform the following steps: determining a position of the surveyingpole with respect to the base station, wherein a location of the basestation is calibrated with respect to an environment; measuring alocation of a first point using the surveying pole based on the positionof the surveying pole with respect to the base station; determining thatthe surveying pole is out of a line of sight of the base station;generating a local map based on images acquired by the camera of thewearable device; determining an offset of the surveying pole withrespect to the local map based on images of the surveying pole acquiredby the camera, wherein the offset is determined after determining thatthe surveying pole is out of the line of sight of the base station;and/or measuring a location of a second point based on the offset of thesurveying pole with respect to the wearable device. In some embodiments,the offset is a second offset; the instructions, when executed, furthercause the one or more processors to determine a first offset of thesurveying pole with respect to the local map based on images of thesurveying pole acquired by the camera; the first offset is determinedbefore determining that the surveying pole is out of the line of sightof the base station; the instructions, when executed, further cause theone or more processors to orient the local map to the environment basedon the position of the surveying pole with respect to the base stationand the first offset of the surveying pole with respect to the localmap; the base station is a robotic total station; the base station isconfigured to not move relative to the environment during measurements;the surveying pole comprises a reflector; the surveying pole comprises aprism; and/or the instructions cause the one or more processors toperform the following steps: calculating cumulative errors over aduration of time while the surveying pole is out of the line of sight ofthe base station; determining the cumulative errors have exceeded athreshold value; indicating to a user of the wearable device that thecumulative errors have exceeded the threshold value; estimatingcoordinates of the surveying pole with respect to the environment, basedon images from the camera while the surveying pole is out of the line ofsight of the base station; and/or sending estimated coordinates of thesurveying pole to the base station. In certain configurations, awearable device comprises a frame; a camera coupled with the frame; andone or more memory devices containing instructions that, when executed,cause one or more processors to perform the following steps: determiningthat a surveying pole is out of a line of sight of a base station;generating a local map based on images acquired by the camera;determining an offset of the surveying pole with respect to the localmap based on images of the surveying pole acquired by the camera,wherein the offset is determined after determining that the surveyingpole is out of the line of sight of the base station; and/or measuring alocation of a point based on the offset of the surveying pole withrespect to the wearable device. In some embodiments, the offset is asecond offset; the instructions, when executed, further cause the one ormore processors to determine a first offset of the surveying pole withrespect to the local map based on images of the surveying pole acquiredby the camera; the first offset is determined before determining thatthe surveying pole is out of the line of sight of the base station;and/or the instructions, when executed, further cause the one or moreprocessors to orient the local map to an environment based on a positionof the surveying pole with respect to the base station and the firstoffset of the surveying pole with respect to the local map.

In certain embodiments, a method for using augmented reality forbuilding layout comprises: determining a position of a surveying polewith respect to a base station, wherein a location of the base stationis calibrated with respect to an environment; measuring a location of afirst point using the surveying pole based on the position of thesurveying pole with respect to the base station; determining that thesurveying pole is out of a line of sight of the base station; generatinga local map based on images acquired by a camera of a wearable device;calculating an offset of the surveying pole with respect to the localmap based on images of the surveying pole acquired by the camera,wherein: the surveying pole can move independently from the camera ofthe wearable device, and the offset is determined after determining thatthe surveying pole is out of the line of sight of the base station;measuring a location of a second point using the surveying pole based onthe offset of the surveying pole with respect to the wearable device;calibrating the location of the base station with respect to theenvironment; calculating cumulative errors over a duration of time whilethe surveying pole is out of the line of sight of the base station;determining the cumulative errors have exceeded a threshold value;indicating to a user of the wearable device that the cumulative errorshave exceeded the threshold value; estimating coordinates of thesurveying pole with respect to the environment, based on images from thecamera while the surveying pole is out of the line of sight of the basestation; sending estimated coordinates of the surveying pole to the basestation; identifying a computer-readable code in the environment usingone or more images acquired by the camera; and/or transmit data aboutthe computer-readable code from the wearable device to the base station.In some embodiments, the offset is a second offset; the method furthercomprises determining a first offset of the surveying pole with respectto the local map based on images of the surveying pole acquired by thecamera; the first offset is determined before determining that thesurveying pole is out of the line of sight of the base station; themethod further comprises orienting the local map to the environmentbased on the position of the surveying pole with respect to the basestation and the first offset of the surveying pole with respect to thelocal map; the surveying pole comprises a prism; the camera is one of aplurality of cameras of the wearable device; and/or images from theplurality of cameras are used to generate the local map and/or determinethe offset of the surveying pole with respect to the local map.

In certain embodiments, a system comprises a light source configured todirect light to form a target on a surface; a wearable device separatefrom the light source, the wearable device comprising a camera and adisplay; and one or more processors configured to: measure a relativelocation of the target to a base station; generate a local map based ona plurality of images acquired by the camera, wherein the local mapincludes a relative location of the target to the wearable device;orient the local map to an environment of the base station based on therelative location of the target to the base station, the relativelocation of the target to the wearable device, and a relative locationof the base station to the environment; and/or present, on the displayof the wearable device, a virtual object in relation to the environmentand/or measure one or more coordinates of a physical object, based onorienting the local map of the wearable device with the environment. Insome embodiments, the light source is a laser; the target is a spot; thetarget is a non-elliptical, two-dimensional design; the base station isseparate from the light source; and/or the one or more processors arefurther configured to: ascertain an orientation of the wearable devicein relation to the environment based on an orientation of a design ofthe target; receive a section of a design of the target from a userusing the wearable device; track an eye of a user, and move a positionof the target based on tracking the eye of the user; and/or move aposition of the target in response to movement of the wearable device.In some embodiments, a system comprises a camera of a wearable device; adisplay of the wearable device; and one or more processors configuredto: measure a relative location of a target to a base station; generatea local map based on a plurality of images acquired by the camera,wherein the local map includes a relative location of the target to thewearable device; orient the local map to an environment of the basestation based on the relative location of the target to the basestation, the relative location of the target to the wearable device, anda relative location of the base station to the environment; present, onthe display of the wearable device, a virtual object in relation to theenvironment and/or measure one or more coordinates of a physical object,based on orienting the local map of the wearable device with theenvironment; and/or transmit an estimated location of the wearabledevice to the base station.

In certain embodiments, a method comprises directing light, from a lightsource, to form a target on a surface; measuring a relative location ofthe target to a base station; generating a local map based on imagesacquired by a camera of a wearable device, wherein the local mapincludes a relative location of the target to the wearable device;orienting the local map to an environment of the base station based onthe relative location of the target to the base station, the relativelocation of the target to the wearable device, and a relative locationof the base station to the environment; presenting, on a display of thewearable device, a virtual object in relation to the environment and/ormeasuring one or more coordinates of a physical object, based onorienting the local map of the wearable device with the environment;ascertaining a depth and/or orientation of a physical object in relationto the wearable device; calculating three-dimensional coordinates of thephysical object in relation to the environment, wherein the local mapincludes a relative location of the physical object to the wearabledevice; calculating the three-dimensional coordinates of the physicalobject is based on the relative location of the physical object to thewearable device, the relative location of the target to the basestation, and the relative location of the bases station to theenvironment; ascertaining an orientation of the wearable device inrelation to the environment based on an orientation of a design of thetarget; tracking an eye of a user; moving a position of the target basedon tracking the eye of the user; receiving, from a user, a selection ofa design; directing light to form the target on the surface, wherein ashape of the target on the surface resembles the design; and/or moving aposition of the target in response to movement of the wearable device.In some embodiments, the light source is a projector; the target is aspot; a location of the wearable device is based on relative position ofthe wearable device to the spot; and/or orientation of the wearabledevice is based on features in the environment imaged by the camera ofthe wearable device.

Further areas of applicability of the present disclosure will becomeapparent from the detailed description provided hereinafter. It shouldbe understood that the detailed description and specific examples, whileindicating various embodiments, are intended for purposes ofillustration only and are not intended to necessarily limit the scope ofthe disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is described in conjunction with the appendedfigures.

FIG. 1 depicts an embodiment of a system for identifying points ofinterest during construction layout.

FIG. 2 depicts an embodiment of a wearable device.

FIG. 3 is a simplified example of a relationship between position dataand a local map.

FIGS. 4-6 are further embodiments of a wearable device.

FIG. 7 depicts another embodiment of a system for identifying points ofinterest during construction layout.

FIG. 8 illustrates a flowchart of an embodiment of a process for usingaugmented reality during construction layout.

FIG. 9 depicts an embodiment of a system for surveying.

FIG. 10 is a simplified example of a relationship between position dataand an offset between a reflector and a wearable device.

FIG. 11 displays a simplified diagram of line of sight to a base stationbeing blocked.

FIG. 12 is an embodiment of displaying cumulative error to a user whileline of sight is blocked to a base station.

FIG. 13 illustrates a flowchart of an embodiment of a process forswitching truth from a surveying pole to a wearable display.

FIG. 14 illustrates a flowchart of an embodiment of a process for usinga wearable display as truth for surveying.

FIG. 15 is a simplified example of using a target on a surface as truthfor measurements.

FIG. 16 is an embodiment of a movable design on a surface used as atarget for truth.

FIG. 17 illustrates a flowchart of an embodiment of a process for usinga target as truth.

FIG. 18 depicts an embodiment of a user marking a position to orient alocal map of a wearable device to an environment.

FIG. 19 depicts an embodiment of a user marking a second position toorient the local map to the environment.

FIG. 20 depicts a block diagram of an embodiment of a computer system.

In the appended figures, similar components and/or features may have thesame reference label. Further, various components of the same type maybe distinguished by following the reference label by a dash and a secondlabel that distinguishes among the similar components. If only the firstreference label is used in the specification, the description isapplicable to any one of the similar components having the same firstreference label irrespective of the second reference label.

DETAILED DESCRIPTION

The ensuing description provides preferred exemplary embodiment(s) only,and is not intended to limit the scope, applicability, or configurationof the disclosure. Rather, the ensuing description of the preferredexemplary embodiment(s) will provide those skilled in the art with anenabling description for implementing a preferred exemplary embodiment.It is understood that various changes may be made in the function andarrangement of elements without departing from the spirit and scope asset forth in the appended claims.

Augmented reality can be used in surveying processes to increaseefficiency in measurement and/or layout. In some embodiments, areflector is coupled with a hardhat and an augmented-reality system. Arobotic total station tracks the reflector to obtain position data ofthe reflector. The augmented-reality system orients itself with anenvironment based on the position data. The augmented-reality systempresents to a user a virtual object (e.g., from a to-be-built model) inrelation to the environment based on orienting itself with theenvironment using the position data from the total station. Theaugmented-reality system can render the virtual objet more accuratelywith the environment based on position data from the total station.Examples of using a reflector coupled with a hardhat are given in FIGS.1-8.

In some embodiments, “truth” can be transferred from a surveying pole tothe augmented-reality system. Truth provides a reference from wheremeasurements or positions are measured from. A surveying pole can beused as truth (e.g., the surveying pole is tracked by a total station,wherein truth of the surveying pole is derived from truth of the totalstation). As a user moves out of line of sight of the total station(e.g., the user goes behind an obstruction), the surveying pole is notused as truth. Instead, truth is transferred to the wearable device byusing position and/or mapping of the augmented-reality system. In someembodiments, position and/or mapping of the augmented-reality system isnot as accurate as using data from the total station, and errors canaccumulate by not using the total station. Accordingly, theaugmented-reality system can track errors and/or present estimatederrors to a user. The augmented-reality system can also transmitestimated position of the surveying pole to the total station so thetotal station can more efficiently re-acquire tracking of the surveyingpole (e.g., after the user emerges from the obstruction). Examples oftransferring truth from a surveying pole to a wearable device are givenFIGS. 9-14.

In some embodiments, a target is used as truth for measurements and/orlayout. A laser (e.g., from a robotic total station), and be used toform a target on a surface. The total station measures a position of thetarget relative to an environment. An augmented-reality system measuresa position of the target relative to the augmented-reality system and/ororients a local map of the augmented-reality system with the environmentbased on both the position of the target relative to theaugmented-reality system and the position of the target relative to theenvironment. Examples of using a target for truth are given FIGS. 15-17.

Though example figures for example embodiments are given above,embodiments are not meant to be mutually exclusive. For example, truthcould be transferred from a hardhat mounted reflector to anaugmented-reality system, or to a target off the hardhat, when a userwearing the hardhat moves behind an obstruction.

Referring first to FIG. 1, an embodiment of a system for identifyingpoints of interest during construction layout is depicted. The systemcomprises a wearable device 104 and a base station 108. The wearabledevice 104 comprises a hardhat 112, a reflector 116, an optical display120, and a camera 124.

The reflector 116, the optical display 120, and the camera 124 arefixedly coupled with the hardhat 112. In some embodiments, fixedlycoupled means attached so as to block relative motion while in use. Insome embodiments, the optical display 120 and the camera 124 are part ofan augmented-reality system. The camera 124 is configured to acquire aplurality of images. A graphics engine (e.g., of the augmented-realitysystem) is configured to generate a local map of the surrounding area,e.g., using a simultaneous localization and mapping (SLAM) algorithm.The wearable device 104 is configured to be worn by the user while inuse.

The base station 108 is configured to measure angles and/or distancesfrom the base station 108 to the reflector 116 to generate position dataof the reflector 116 in relation to an environment. In some embodiments,the base station 108 is a robotic total station. The base station 108can include a theodolite, an image sensor, a laser having a visiblewavelength, and/or an electronic distance measurement (EDM) unit. Forexample, angles can be measured from the base station 108 using atheodolite and/or an image sensor. Distances from the base station 108to the reflector 116 can be measured using an EDM unit. Distances fromthe base station 108 to the reflector 116 can be measured by a positionof the reflector 116 on an image sensor in relation to a camera center.Distances from the base station 108 to the reflector 116 can be measuredusing a length of the reflector 116 on the image sensor, a known actuallength of the reflector 116, and/or a focal length of optics used tofocus an image of the reflector 116 on the image sensor. Position datacan include location data (e.g., x, y, and z data of the wearable device104 the base station 108) and/or orientation data of the wearable device104. Position data can be generated by the base station 108 and/or thewearable device 104. For example, location data can be generated by thebase station 108 and orientation data can be generated by the wearabledevice 104. The base station 108 can be configured to remain stationaryduring measurements.

The base station 108 is oriented to the environment. For example, thebase station 108 is oriented to known point 130. In some embodiments,the base station 108 can be oriented to the environment by identifyinglocations of known targets and/or features of the environment.

A reflector 116 can include a prism, foil, a sticker, and/orretroreflector (e.g., a reflector that reflects light back toward asource for a wide range of incident angles, such as a corner reflectoror cat eye reflector). A reflector 116 can be used in conjunction withan active target (e.g., LED light(s)) and/or a passive target (e.g.,design, contrasting colors, etc.).

One or more processors are configured to receive position data; orientthe local map of the wearable device 104 to the base station 108 usingthe position data; and/or present to a user of the wearable device 104,using the optical display 120, a virtual object in relation to theenvironment based on orienting the local map of the wearable device 104with the base station 108. The one or more processors can be part of thewearable device 104, the base station 108, and/or a separate device,such as a computer, mobile device, and/or smart phone.

The virtual object can come from a model. The virtual object can be oneor more points of interest from the model. For example, the model caninclude anchor points, and positions of the anchor points can be shownto the user, as virtual objects, by the optical display 120. By showingthe user a virtual object in relation to the environment, constructiontasks can be simplified. In the example above, a construction worker cansee, through the optical display 120, a virtual object (e.g., a dot)where an anchor point is to be located on a physical wall.

In some embodiments, a wearable device comprises a frame (e.g., hardhat112, glasses frame, head-mounted display, hat, cap, helmet, ear muffs,and/or headband); a marker (e.g., a reflector, a light, a design, and/orone or more marks); an optical display (e.g., an augmented-realitydisplay); a camera; a graphics engine; and/or one or more processors.The marker can be fixedly coupled with the frame; the optical displaycan be fixedly coupled with the frame; and/or the camera can be fixedlycoupled with the frame. The graphics engine is configured to provide alocal map based on images acquired by the camera (e.g., using a SLAMalgorithm). One more processors are configured to receive position data;orient the local map of the wearable device to the base station usingthe position data; and/or present, using the optical display, a virtualobject in relation to the environment. For example, a location of avirtual anchor point is shown in relation to a physical wall, floor,and/or ceiling. The base station 108 can measure position and/ororientation of the marker in relation to the base station 108. Inembodiments having the optical display fixedly attached to the frame,and the marker also fixedly attached the frame, a known position and/ororientation of the optical display can be known in relation to the basestation 108 because there is no movement between the wearable device andthe marker.

FIG. 2 depicts an embodiment of the wearable device 104. The wearabledevice 104 comprises a hardhat 112, a reflector 116, an optical display120, and a camera 124. The optical display 120 allows real-world light204 to pass through the optical display 120. The optical display 120 ispart of an augmented-reality system. The augmented-reality systempresents a virtual object to a user of the wearable device 104 using theoptical display 120 (e.g., by using a holographic waveguide or apolarized waveguide). In some embodiments, the wearable device 904comprises a plurality of cameras 124 (e.g., as part of anaugmented-reality system). A first camera 124-1 and a second camera124-2 are shown on a right side of an augmented-reality system. Theaugmented-reality system can comprise more cameras. Other cameras 124can be placed at a left side, a center (e.g., “between the eyes”), back,and/or other areas of the augmented-reality system and/or the housing208. Cameras 124 can have different fields of view. For example, thefirst camera 124-1 has a first field of view and the second camera 124-2has a second field of view, wherein the second field of view is widerthan the first field of view. The plurality of cameras 124 can have acombined field of view (e.g., a field of view equal to or greater than75, 90, 180, or 210 degrees, and/or equal to or less than 90, 180, 210,270, or 360 degrees).

The reflector 116 is a prism. Other types of reflectors or targets couldbe used. The hardhat 112 is designed to be impact resistant. Forexample, the hardhat 112 is configured to comply with American NationalStandards Institute (ANSI) standard for head protection, Z89.1 (2009) orthe Canadian standards Association (CSA) Industrial Protective Headwear,Z94.1 (2005).

The wearable device comprises a housing 208. The housing 208 couples theoptical display 120 and/or the camera 124 to the hardhat 112. Thehousing 208 comprises a battery compartment 212 and/or one or moreprocessors. The one more processors can be used to carry out processingtasks. In some embodiments, the housing 208 can be referred to as aframe. In some embodiments, the housing 208 is securely attached to thehardhat 112 and configured to not be removable by an end user (e.g.,using adhesive or an inseparable snap fit) or not be removable by an enduser without tools (e.g., using screws). In some embodiments, thehardhat 112 is configured so that the housing 208 is removably attachedto the hardhat 112 (e.g., by the hardhat 112 having a separable snap-fitcantilever mechanism that secures the housing 208 to the hardhat).

In some embodiments, the wearable device 104 includes an inertialmeasurement unit (IMU), which can be used to provide orientation data(e.g., direction relative to force of gravity). The IMU can beintegrated with the housing 208 and/or an augmented-reality system 128comprising the one or more cameras 124 and the optical display 120.

FIG. 3 is a simplified example of a relationship between position dataand a local map. The base station 108 tracks (e.g., visually) thewearable device 104 to determine a position of the wearable device 104in relation to the base station 108 to obtain position data.

Position data can include a distance 304 between the base station 108and the wearable device 104 and/or relative angle(s) (e.g., azimuthand/or altitude) of the wearable device 104 to the base station 108. Insome embodiments, the position data includes an orientation of thewearable device 104 in relation to the base station 108.

FIG. 3 depicts a space 308. The space 308 is an area or a volume of thelocal map. The space 308 includes features 312, such as surfaces, edges,and/or corners that have been in a field of view of one or more cameras124 of the wearable device 104. The local map can include relativedistances and/or orientations between the features of the local map. Insome embodiments, the local map can include orientation data of thewearable device 104 in relation to one or more features 312. Forexample, a feature 312 can include a corner or a horizontal line formedby an intersection of a wall and a ceiling. The local map coulddetermine orientation of the wearable device 104 relative to theenvironment based on relative orientation to the corner, horizontalline, vertical line, or some other line of known orientation. In someembodiments, the base station 108 can measure a distance and/or adirection to a feature 312 of the environment. Accordingly, positiondata can include location data and/or orientation data measured to afeature 312 of the local map. In some embodiments, the distance 304 ismeasured using an electronic distance measurement (EDM) device.

The Environment 316 is a physical space (e.g., a three dimensionalspace) and includes physical objects within the physical space. Thelocal map is oriented to the environment 316. Examples of theenvironment 316 include a building, a floor of a building, aconstruction site, a road way, a parking lot, a field, a dwelling, anoffice space, a yard, a room, etc.

While a local map has a given precision, the given precision may not besufficient for certain applications. For example, if the given precisionof the local map is 20 cm and a desired precision for a constructionlayout is 10 cm or better, then the given precision of the local map isinsufficient. Precision of the local map can also degrade over largerareas. For example, precision of the local map degrades as aconstruction worker walks 20 yards from a first location to a secondlocation. The local map might not keep enough data points to accuratelytrack relative position over the distance of 20 yards. To improveprecision and/or accuracy, a robotic total station can be used (e.g., asbase station 108) to accurately determine a position of the wearabledevice 104 within the environment 316. The local map is calibrated tothe wearable device 104. By calibrating the local map to the wearabledevice 104, and the wearable device 104 to the environment 316, theprecision and/or accuracy of the local map can be increased.

Points 320 are virtual objects generated by the optical display. Thepoints 320 are generated by the optical display and presented to a userof the wearable device 104 to appear in relation to the environment 316.Accordingly, the points 320 appear to the user to be on a wall in theenvironment 316, but another person standing next to the user of thewearable device 104 would not see the points 320. Though points 320 areshown as the virtual objects, other virtual objects can be shown. Forexample rendering of plumbing, HVAC, electrical, structural, and otheritems could be shown as virtual objects in relation to the environment316 (e.g., rendering of to-be-built objects).

FIGS. 4-6 are further embodiments of a wearable device. FIG. 4 depictsan embodiment of a first wearable device 104-1; FIG. 5 depicts anembodiment of a second wearable device 104-2; FIG. 6 depicts anembodiment of a third wearable device 104-3. An EDM and/or a camera canbe used at the base station measure a distance from the base station tothe wearable device 104 and/or an orientation of the wearable device. Insome embodiments, an EDM is co-centered with a camera at the basestation.

In FIG. 4, the first wearable device 104-1 has reflective tape 404. Thereflective tape 404 can be retroreflective tape. The reflective tape 404is placed on the hardhat in a pattern, e.g., a known pattern so thatorientation of the hardhat can be determined by an image of the pattern.In some embodiments, the reflective tape 404 is a first reflector inaddition to a second reflector on the hardhat (such as the prism shownas reflector 116 in FIG. 2), another piece of reflective tape, or foil.The second reflector can be in a known orientation to the firstreflector (e.g., so that an orientation of the first wearable device104-1 can be determined based on an image of the first reflector, thesecond reflector, and known relative orientations of the first reflectorto the second reflector). The reflective tape 404 is a type of marker.

In FIG. 5, the second wearable device 104-2 has a code 504 on thehardhat. The code 504 is a machine-readable code in the form of apattern. Examples of code 504 can include a bar code (e.g., a pattern oflines with varying space and/or thicknesses); a Quick Response (QR) code(e.g., a 2D barcode); ruler markings; and two-dimensional patterns, withor without characters. The code 504 is fixedly attached to the hardhatand/or formed in the hardhat (e.g., etched after the hardhat is made, ormolded with the hardhat as the hardhat is being made). The code 504 is atype of marker. In one example, a first code 504-1 (e.g., a ruler or abar code) is orientated vertically on the hardhat, and a second code504-2 (e.g., a ruler or a bar code) is oriented horizontally on thehardhat. The first code 504-1 and the second code 504-2 have uniquemarkings. A base station images the second wearable device 104-2 and candetermine an orientation of the wearable device based on how the firstcode 504-1 and/or the second code 504-2 face the base station. The basestation can also determine a distance from the base station to thewearable device based on known distances between markings of the code504 imaged on an image sensor.

In FIG. 6, the third wearable device 104-3 has a design 604. The design604 can be a two-dimensional design or a three dimensional design. Forexample, the design 604 can be a target covering a portion of an outsidesurface area of the hardhat (e.g., covering an area equal to or lessthan 50, 35, 25, 20, 10, or 5 percent and/or covering an area equal toor greater than 5, 10, 15, or 20 percent). In some embodiments, thedesign 604 could be an artistic design. For example, a design of a flag,animal, dragon, logo, geometric shape, etc. could be used. Similarly asin FIG. 5, dimensions of the design 604 can be known to enable the basestation to determine a location and/or orientation of the third wearabledevice 104-3. The design 604 is a marker.

FIG. 7 depicts another embodiment of a system for identifying locationsof points of interest during construction layout. The system comprises acamera 704, along with a wearable device 104 and/or a base station 108.The camera 704 is a tracking camera. The camera 704 is separate from thebase station 108 and the wearable device 104. In some embodiments, thecamera 704 is part of the base station 108. In some embodiments, thecamera 704 is part of the wearable device 104 and tracks relativemovement between the wearable device 104 and the base station 108 orother feature calibrated to the environment.

The camera 704 acquires image data (e.g., pictures) used to determine aposition of the wearable device 104 within the environment. For example,a robotic total station is used as the base station 108. The robotictotal station is used to precisely determine a location of the camera704 in the environment. The camera 704 can be used to track the wearabledevice 104 as a user moves about the environment. Having the camera 704separate from the robotic total station can be useful in severalsituations. For example, the total station might not have line of sightto the wearable device 104, but the camera 704 does (e.g., a feature inthe environment, such a as a post, wall, or tree blocks line of sightfrom the total station to the wearable device 104). In another example,the camera 704 could be placed so that the wearable device 104 is notsilhouetted by a relatively bright light source (e.g., a user standingby a window).

In, FIG. 8, a flowchart of an embodiment of a process 800 for usingaugmented reality during construction layout is shown. Process 800begins in step 804 with measuring a position of a reflector in relationto a base station to obtain position data. The reflector is fixedlycoupled with a hardhat, and the base station is configured to measureangels and distances from the base station to the reflector in relationto an environment (e.g., as described in conjunction with FIG. 1). Instep 808, a local map is generated based on images acquired by a camerathat is fixedly coupled with the hardhat. For example, camera 124 of anaugmented-reality system acquires images, and a SLAM algorithm is usedto create the local map based on the acquired images.

In step 812, the local map is oriented to an environment (e.g., byorienting the local map the base station) using the position data. Forexample, the base station is oriented to the environment (e.g., bysurveying techniques), and the base station measures a distance and/oran angle from the base station to the wearable device (e.g., positiondata). The local map is created in relation to the wearable device.Accordingly, the local map can be oriented to the environment based onthe distance and/or the angle of the wearable device to the basestation.

In some embodiments, a model in relation to the environment is saved inone or more memory units of the user device, the base station, a mobiledevice (e.g., a tablet, laptop, or smartphone) and/or other computingdevice. The model can contain information, such as points and/or athree-dimensional representation of a change to the environment (e.g.,plumbing to be added, a change in grade, framing, finishing, appliances,architecture changes, etc.). The model, or a portion of the model, ispresented to the user of the wearable device by the optical display asone or more virtual objects in relation to the environment. The virtualobject appears in proper relation to the environment based on orientingthe local map of the augmented-reality display with the base station.

The process 800 can include more or fewer steps. For example, process800 can further include orienting the model to the environment and/ororienting the base station to the environment (e.g., creating abaseline), wherein orienting the model to the environment and/ororienting the base station to the environment can be done before 804. Insome embodiments, the base station can be outside of the environment.For example, the base station could be a distance of 10 or 50 yards awayfrom the environment that the user is operating in.

Though construction layout is provided as an example, the example is notmeant to be a limiting example, and the system and/or techniques can beused in other applications (e.g., navigation, entertainment, medical,repair, design, modeling, as-built verification, etc.). The systemand/or techniques can be used in modeling, such as visualization of newconstruction. For visualization of new construction, one particularlybeneficial application is visualization walk through when there are verylittle physical features. For example, a home builder could walk a buyerthrough a virtual home when there is only a foundation of the homeconstructed. Using traditional SLAM techniques, the local map might havea difficult time orienting to the environment of just a foundation. Forexample, if a user looked up to “see” where a ceiling was to be placed,the SLAM algorithm may have difficulty identifying features becausethere could be no physical features present as the user looks at thesky. However, by using the base station to orient the local map to theenvironment, the augmented-reality display could generate a virtualimage of a model of the home to be built in relation to the foundationof the home. The user could switch between different models (e.g., of ahome) while standing near or on the foundation (or standing at an emptyfield).

Referring next to FIG. 9 an embodiment of a system for surveying isshown. The system comprises a wearable device 904, a base station 108,and a surveying pole 916. A reflector 116 is coupled with the surveyingpole 916.

The wearable device 904 can be similar to the wearable device 104 ofFIG. 2 (e.g., the wearable device 904 can comprise one or more cameras124, an optical display 120, a hardhat 112, a housing 208, a frame, abattery compartment 212, one or more processor, one or more memorydevices, and/or an augmented-reality system 128). One difference betweenthe embodiments shown in FIG. 1 and FIG. 9 is that the reflector 116 ordesign (e.g., 404, 504, and 604 in FIGS. 4-6) in FIG. 9 is coupled withthe surveying pole 916 instead of the wearable device 104 in FIGS. 2 and4-6.

The base station 108 is used as “truth” during surveying while there isa direct line of sight between the reflector 116 and the base station108. “Truth” refers a reference from which a surveying point is measuredrelative to. For example, during measurements while there is a directline of sight between the reflector 116 and the base station 108, surveypoints are measured with respect to the base station 108, because adistance between the base station 108 and the reflector 116 is measuredby the base station 108, and a relative position between the reflector116 and a tip 920 of the surveying pole 916 is known (or ascertainable).

Sometimes, a line of sight between the base station 108 and thereflector 116 can be blocked. For example, a pillar, a wall, a piece ofequipment, or the user can block the line of sight between the basestation 108 and the reflector 116. While the line of sight is blocked tothe base station 108, some traditional surveying systems cannot continueto perform measurements. In some embodiments, “truth” is transferredfrom the base station 108 to the wearable device 904 while line of sightbetween the reflector 116 and the base station 108 is blocked. Thusmeasurements made while the wearable device 904 is “truth,” are madewith respect to an offset between the surveying pole 916 and thewearable device 904.

In some embodiments, one or more memory devices containing instructionsthat, when executed, cause one or more processors to perform thefollowing steps: determining that a surveying pole 916 is out of a lineof sight of a base station; generating a local map based on imagesacquired by the camera; determining an offset of the surveying pole withrespect to the local map based on images of the surveying pole acquiredby the camera, wherein the offset is determined after determining thatthe surveying pole is out of the line of sight of the base station;and/or measuring a location of a point based on the offset of thesurveying pole with respect to the wearable device. The base station 108can be configured to not move location relative to the environmentduring measurements.

FIG. 10 is a simplified example of a relationship between a distance1004 between the reflector 116 and the base station 108 and an offset1008 between the reflector 116 and a wearable device 904. The offset1008 is a distance between the wearable device and the surveying pole916 (e.g., measured from the reflector 116 to the wearable device 904 bythe wearable device 904 using an augmented-reality system of thewearable device 904). The offset 1008 can be measured using an algorithmof the wearable device 904 (e.g., a SLAM algorithm).

The base station 108 is oriented with an environment 1010 (e.g., usingsurveying techniques to place a robotic total station at a constructionsite) to calibrate the base station 108 with respect to the environment1010. A position of the surveying pole 916 is determined with respect tothe base station 108 by measuring a distance 1004 between the basestation 108 and the reflector 116 and/or measuring one or more angles(e.g., measuring an angle α) between the base station 108 and thereflector 116. The distance 1004 and/or the one or more angles can beused to measure a point 1012 with respect to the base station 108.

A local map of a space 1016 can be generated by images from one or morecameras of an augmented-reality system of the wearable device 904. Thespace 1016 includes features 312 and the reflector 116. The local mapincludes relative orientation of the reflector 116 with respect thewearable device and the offset 1008. By using the relative orientationof the reflector 116 with respect to the wearable device, the offset1008, the distance 1004, and/or the one or more angles (e.g., angle α),the local map can be calibrated to the environment 1010.

In some embodiments, the wearable device 904 can be used to calibratethe base station 108 to the environment 1010. For example, the wearabledevice 904 can be used to read a machine-readable code 1020 or identifya particular feature with known coordinates of the environment 1010. Thewearable device 904 can determine a position of the machine-readablecode 1020 and relative orientation of the machine-readable code inrelation to the wearable device 904. The wearable device 904 is alsoused to measure the offset 1008 and/or relative orientation of thereflector 116. The base station 108 measures the distance 1004 and/orone or more angles from the base station to the reflector 116. The basestation 108 can be calibrated to the environment 1010 based on the knowncoordinates of the machine-readable code 1020, the relative position ofthe readable code in relation to the wearable device 904, the offset1008 and relative position of the reflector 116 in relation to thewearable device 904, the distance 1004, and/or the one or more anglesmeasured from the base station 108 to the reflector 116. The wearabledevice 904 can be used to read and/or transmit data of themachine-readable code 1020 to the base station 108. Thus the wearabledevice 904 can be used to help in setting up the base station 108.

A virtual object 1024 can be shown to a user of the wearable device 904in relation to the environment 1010. For example, virtual object 1024does not exist, but appears to the user to be on the wall above themachine-readable code 1020.

FIG. 11 displays a simplified diagram of transferring truth from asurveying pole 916 to a wearable device 904 while line of sight to thebase station 108 is blocked. FIG. 11 depicts a user making threedifferent measurements of three different points at three differenttimes. At a first time, the surveying pole 916 is that a first point1104-1; at a second time, the surveying pole 916 is at a second point1104-2; and at a third time, the surveying pole 916 is at a third point1104-3. The user moves the surveying pole 916 from the first point1104-1 to the second point 1104-2, and from the second point 1104-2 tothe third point 1104-3.

The base station 108 has line of sight with the surveying pole 916 whilea tip of the surveying pole 916 is placed at the first point 1104-1 andwhile the tip of surveying pole 916 is placed at the third point 1104-3.However, an obstruction 1108 blocks line of sight from the base station108 to the surveying pole 916 while the tip of the surveying pole 916 isat the second point 1104-2. The base station 108 is used as truth formeasuring the first point 1104-1 and the third point 1104-3; thewearable device 904 is used as truth for measuring the second point1104-2. Measurements using the base station 108 often have higherprecision than measurements using the wearable device 904. Accordingly,the base station 108 is used as truth while there is line of sightbetween the base station 108 and the surveying pole 916, thoughmeasurements using the wearable device 904 as truth could be performedin situations with more relaxed precision tolerances.

The following is an example of taking measurements while transferringtruth from the base station 108 to the wearable device 904 and then backto the base station 108. A user calibrates the base station 108 with anenvironment and calibrates a local map of the wearable device 904 to theenvironment (e.g., as described in FIG. 10). The user places the tip ofthe surveying pole 916 at the first point 1104-1. The distance betweenthe tip of the surveying pole 916 and the reflector is known. The basestation 108 measures a first distance 1004-1 from the base station 108to the reflector of the surveying pole 916 and/or relative anglesbetween the base station 108 and the surveying pole 916. The location ofthe first point 1104-1 is measured based on the first distance 1004-1and/or relative angles between the base station 108 and the surveyingpole 916 (i.e., the base station 108 is used as truth for measuring thelocation of the first point 1104-1).

The user moves the surveying pole 916 from the first point 1104-1 to thesecond point 1104-2. As the user moves the surveying pole to the secondpoint 1104-2, the obstruction 1108 blocks line of sight of the basestation 108 to the surveying pole 916. The surveying pole 916 can bedetermined to be out of the line of sight of the base station 108 invarious ways. For example, the base station 108 could determine that thebase station 108 is no longer tracking a part of the surveying pole(e.g., reflector) and/or send a wireless signal to the wearable device904. In another example, a reflector coupled with the surveying polecould detect light received from the base station 108 (e.g., thereflector can detect light from an EDM of the base station 108), andwhen the reflector no longer detects light received from the basestation 108, the reflector could activate a signal (e.g., activating alight source having a particular wavelength and/or duty cycle, such as ared LED flashing two times per second). The wearable device 904 couldidentify the signal and ascertain that the base station 108 is no longerin line of sight with the surveying pole 916.

After determining that the base station 108 is no longer in line ofsight with the surveying pole 916, an offset of the surveying pole 916with respect to the local map of the wearable device 904 (e.g., anoffset as 1008 and/or relative position of the reflector 116 withrespect to the wearable device 904 as described in FIG. 10) isdetermined (e.g., ascertained), based on images acquired by one or morecameras of the wearable device 904. A location of the second point1104-2 is measured based on the offset of the surveying pole withrespect to the wearable device (i.e., the wearable device 904 is used astruth for measuring the location of the second point 1104-2). Since thewearable device 904 is used as truth to measure the location of thesecond point 1104-2 with respect to the environment, truth is said to betransferred from the base station 108 to the wearable device 904.

As the user walks from the second point 1104-2 to the third point1104-3, truth is transferred from the wearable device 904 to the basestation 108 as the surveying pole 916 comes into line of sight of thebase station 108. The user places a tip of the surveying pole 916 at thethird point 1104-3. The base station 108 measures a second distance1004-2 from the base station 108 to the surveying pole 916 and/orrelative angles from the base station 108 to the surveying pole 916. Thelocation of the third point 1104-3 is measured based on the seconddistance 1004-2 and/or relative angles between the base station 108 andthe surveying pole 916 (i.e., the base station 108 is used as truth formeasuring the location of the third point 1104-3).

Transferring truth from the base station 108 to the wearable device 904can also be done in conjunction with other embodiments disclosed herein.For example, in the embodiment in FIG. 3, if a line of sight between thebase station 108 and the wearable device 104 were blocked, truth couldbe transferred to the wearable device 104 for the user to continue totake measurements.

Measurements using the base station 108 as truth are generally moreprecise than measurements using the wearable device 904 as truth. Onereason the wearable device 904 may not have as precise measurements isbecause a calculation of relative position of the wearable device 904 tothe environment 1010 can drift, or accumulate errors, over time. Thus iscan be preferred to use the base station 108 as truth forhigher-precision applications. In some embodiments, the optical display120 can show cumulative error while the wearable device 904 is used astruth (the optical display 120 can also show estimated error while thebase station 108 is used as truth).

The wearable device 904 can be used to help the base station 108 trackthe surveying pole 916. A robotic total station can be configured totrack a reflector. However, if line of sight between the robotic totalstation and the reflector is obscured, then the robotic total stationimplements an algorithm for acquiring the reflector. Some algorithmsinclude waiting for a certain amount of time and then entering a searchroutine where the robotic total station scans the environment searchingfor the reflector (e.g., a raster-type scan or circular scan). Scanningthe environment searching for the reflector can take time away frommaking measurements using the robotic total station.

The wearable device 904 can be used to help the base station 108 morequickly acquire tracking the reflector after the reflector is obscuredfrom a line of sight of the base station 108. For example, the wearabledevice 904 measures a relative position of the reflector of thesurveying pole 916 with respect to the wearable device 904. As the usermoves from the first point 1104-1 to the second point 1104-2, theobstruction 1108 blocks line of sight between the base station 108 andthe surveying pole 916, and truth is transferred from the base station108 to the wearable device 904. The wearable device 904 can calculate aposition of the surveying pole 916 in relation to the environment basedon the relative position of the reflector with respect to the wearabledevice 904. The wearable device 904, or some other device such as asmart phone or a tablet, can transmit the position of the surveying pole916 in relation to the environment to the base station 108. Thus thebase station 108 can orient itself to be ready to acquire tracking ofthe surveying pole 916 has a user emerges from behind the obstruction1108 on the way to the third point 1104-3. Without receiving informationabout the position of the surveying pole 916 in relation to theenvironment, the base station 108 would not know if the surveying pole916 is going to emerge to the left, right, up, down, etc. of theobstruction 1108. By sending data about the position of the surveyingpole 916 in relation to the environment, the base station 108 can beprepared to acquire the reflector of the surveying pole 916 as the useremerges from behind the obstruction 1108. In some embodiments, positiondata of the surveying pole 916 (and/or reflector or other target) issent periodically to the base station 108 (e.g., once every 0.25, 0.5,1, 2, 5, or 10 seconds).

FIG. 12 is an embodiment of displaying estimated error. Estimated erroris shown to a user while line of sight is blocked to the base station.In FIG. 12, an embodiment of a user perspective of the environment 1010in FIG. 10 through the optical display 120 is shown. The optical display120 comprises text 1204. The text 1204 indicates the estimated error. Anerror while using the base station can be relatively low (e.g., equal toor less than 1, 2, 5, or 10 mm). After truth is transferred from thebase station 108 to the wearable device 904, cumulative error canincrease as sensors drift. As cumulative error increases, the text 1204can change to indicate increased error in making measurements. Forexample, the text in FIG. 12 shows “±2.0 cm” to indicate to the userthat measurements being made have accuracy likely no better than 2centimeters. The estimated error can increase to a threshold value.After the estimated error reaches or exceeds the threshold value, thewearable device can be configured to stop making measurements. In someembodiments, the threshold value can be set by the user or anadministrator (e.g., the threshold value is set by a foreman to verifyaccuracy of measurements being made).

Truth changing from the base station to the wearable device can becommunicated to the user in various ways. For example, an icon of a basestation can be shown to the user by the optical display 120 (e.g., neartext 1204) to indicate the base station is being used as truth; and anicon of a wearable device can be shown to the user by the opticaldisplay 120 to indicate the wearable device is being used as truth. Inanother example, color of the text 1204 is changed (e.g., green for thebase station as truth and red for the wearable device as truth). In afurther example, the wearable device can display error bounds as acircle, dot, or target to the user at the point being measured. Theexamples above are not limiting or mutually exclusive, and examples canbe combined in various ways.

FIG. 13 illustrates a flowchart of an embodiment of a process 1300 forswitching truth from a surveying pole to a wearable display. Process1300 begins in step 1304 with determining a position of a surveying polewith respect to a base station, wherein a location of the base stationis calibrated with respect to an environment. For example, a location ofthe base station is calibrated with respect to the environment 1010, andthe distance 1004 and/or angles (e.g., angle α) are measured between thebase station 108 and the reflector 116, as described in conjunction withFIG. 10.

A location of a first point using the surveying pole is measured basedon the position of the surveying pole with respect to the base station,step 1308. For example, the location of the first point 1104-1 ismeasured with respect to the base station 108 in FIG. 11.

In step 1312, a determination is made that the surveying pole is out ofa line of sight of the base station. For example, the user in FIG. 11moves the surveying pole 916 behind the obstruction 1108, and the basestation 108 and/or the wearable device 904 determine there is no longera line of sight between the base station 108 and the surveying pole 916.

A local map is generated based on images acquired by a camera of thewearable device 904, step 1316. For example, one or more cameras ofwearable device 904 in FIG. 11 are used to generate a local map, whichcould include features of the obstruction 1108. The local map can beoriented to the environment while there is a line of sight between thesurvey pole 916 and the base station 108.

In step 1320, an offset of the surveying pole with respect to the localmap is calculated based on images of the surveying pole acquired by thecamera. The surveying pole can move independently from the camera of thewearable device, and/or the offset is determined after determining thatthe surveying pole is out of the line of sight of the base station.

A location of a second point using the surveying pole is measured basedon the offset of the surveying pole with respect to the wearable device.For example, the wearable device 904 in FIG. 11 is used as truth formeasuring the second point 1104-2. Truth can be switched back to thebase station. For example, as the user emerges from the obstruction 1108and measures the third point 1104-3, in FIG. 11, the base station 108 isused as truth because there is a line of sight between the surveyingpole 916 and the base station 108 while the surveying pole 916 is at thethird point 1104-3.

In some embodiments, the method comprises calibrating a location of thebase station with respect to the environment. For example, the basestation 108 in FIG. 10 is calibrated to the environment 1010 beforemeasurements are made using the base station 108. The offset can be asecond offset (e.g., an offset of the surveying pole 916 with respect tothe wearable device 904 while the surveying pole 916 is at the secondpoint 1104-2 in FIG. 11), and the method can further comprisedetermining a first offset of the surveying pole with respect to thelocal map based on images of the surveying pole acquired by the camera;the first offset is determined before determining that the surveyingpole is out of the line of sight of the base station; and/or orientingthe local map to the environment based on the position of the surveyingpole with respect to the base station and the first offset of thesurveying pole with respect to the local map. For example, the firstoffset can be between the surveying pole 916 and the wearable device 904while the surveying pole 916 is at the first point 1104-1 in FIG. 11;and the local map is oriented to the environment based on the firstoffset and the first distance 1004-1

Errors (e.g., cumulative errors) can be calculated over a duration oftime while the surveying pole is out of line of sight of the basestation. For example, while the surveying pole 916 is behind theobstruction 1108, cumulative errors are calculated. The wearable devicecan determine the cumulative errors have exceeded a threshold value. Forexample, 10 cm could be a threshold value, and the wearable device couldascertain that the error exceeds 10 cm by comparing the cumulative errorto the threshold value. The wearable device can indicate to a user thatthe error has exceeded the threshold value. For example, the text 1204in FIG. 12 could start blinking the threshold value after the thresholdvalue has been exceeded, and/or an error message could be displayedusing the optical display 120.

Coordinates of the surveying pole with respect to the environment can beestimated, based on images from the camera while the surveying pole isout of line of sight of the base station, and/or estimated coordinatesof the surveying pole can be sent to the base station. For example,while the surveying pole 916 is behind the obstruction 1108 in FIG. 11,the wearable device can estimate a position of the surveying pole 916and/or transmit estimated position(s) of the surveying pole 916 to thebase station 108. The base station 108 can continue to aim at theestimated position of the surveying pole 916 to more quickly acquire thesurveying pole 916 as the surveying pole 916 emerges from theobstruction 1108.

FIG. 14 illustrates a flowchart of an embodiment of a process 1400 forusing a wearable display as truth for surveying. Process 1400 begins instep 1404 with determining that a surveying pole is out of a line ofsight of a base station. A local map is generated using a wearabledevice (e.g., wearable device 904 in FIG. 11) based on images acquiredby one or more cameras of the wearable device, step 1408. In step 1412,an offset of the surveying pole with respect to the local map isdetermined based on images of the surveying pole acquired by the one ormore cameras of the wearable device. The offset is determined afterdetermining that the surveying pole is out of the line of sight of thebase station. A location of a point is measured (e.g., the second point1104-2) based on the offset of the surveying pole with respect to thewearable device, step 1416.

In some embodiments, the wearable device can assist the base station.For example, the wearable device 904 in FIG. 10 can help the basestation 108 scan a surface by providing information about the surfacefrom the local map. For example, the base station can be configured toshine a laser spot on a surface (e.g., for marking a place for an anchorpoint). However, if the surface is not where expected (e.g., the surfaceis an inch higher than planned) and/or the surface is not smooth (e.g.,the surface is undulated), then the base station can have difficultymarking the surface with the laser spot. Yet using information from thelocal map on surface shape, the base station can predict a betterstarting point to shine the laser.

In other examples of the wearable device assisting in construction, thewearable device can be used to confirm a model and/or shape of thereflector. Sometimes a user selects a wrong model of reflector used formeasurement, which can cause measurement errors. The wearable device canacquire one or more images of the reflector and confirm the correctreflector (e.g., model, size, shape, orientation, etc.) is being usedfor calculations. The wearable device can provide as-built verificationof a construction site by recording images. Images for as-builtverification can be logged by date/time (e.g., to log a time when a taskis completed), and/or images can be identified by location and/ororientation based on recorded location and/or orientation of thewearable device.

Images for as-built verification can also be used to identifydifferences between a model and actual completion. The wearable devicecan confirm a correct object and/or identify an object to be used. Forexample, if a user holds a 2 inch anchor where a 2.5 inch anchor is tobe installed, the wearable device can alert the user and/or identify thecorrect 2.5 inch anchor in a bin (e.g., “ERROR” is displayed on theoptical display 120 in FIG. 12, the 2 inch anchor is highlighted in red,and/or the 2.5 inch anchor is highlighted in green (or an arrow isdisplayed to indicate the user is to turn his head to find the correctanchor).

In another example, the wearable display can be used to find objects.For example, the wearable device logs locations of tools. The user couldask a digital assistant, “where is my hammer?” The optical display couldthen indicate an arrow for a direction for the user to turn and/orhighlight a location of the hammer, even if the hammer was obscured(e.g., by a board the user placed on top of the hammer).

The wearable device can access a catalog of objects to help the useridentify a correct object for a given purpose or given location. In someembodiments, the wearable device highlights a next object for use in aproject. For example, if a user is building a shed from a kit, thewearable device could highlight the next board to be used, the nextbrace to be used, and/or a virtual image of the location/orientation ofthe board and brace in relation to a floor already assembled (e.g., tospeed assembly of the shed).

In some embodiments, a method comprises generating a local map of aspace based on images acquired by a camera of a wearable device, whereinthe local map includes surface information of an area in the space;transmitting the surface information of the area to a base station;calculating a starting point in the area to direct a laser based on thesurface information; receiving an identifier of a reflector to be used;acquiring one or more images of the reflector; comparing the identifierof the reflector to the one or more images of the reflector to confirmthe correct reflector is being used; recording images of theenvironment; providing as-built verification based on recorded images(e.g., identifying differences between a virtual model and actualobjects in an image; log time, position, and/or view when complete;etc.); confirming a correct object is installed at a location (e.g., acorrect size of hardware, such as a bolt of the correct length can beconfirmed to be installed at a specific location); and/or identifying anobject; highlighting the object using the display; and/or identifying anext object for a user to touch or pick up.

FIG. 15 is a simplified example of an embodiment using a target 1504 ona surface as truth for measurements. A system comprises a light source1508 and a wearable device 904. The wearable device 904 comprises acamera and an optical display (e.g., a camera and an optical display aspart of an augmented-reality system). The light source 1508 isconfigured to direct light to form the target 1504 on a surface (e.g.,on a wall, floor, ceiling, pillar, etc.). The light source 1508 cancomprise a laser (e.g., coherent light) and/or a projector (e.g., anincoherent light source and a mask). The light source 1508 can be partof a base station 108 or separate from the base station 108. In theexample shown in FIG. 15, the light source 1508 is separate from thebase station 108. In some embodiments, the light source 1508 is a laserthat is part of a robotic total station.

One or more processors (e.g., a part of the wearable device 904, thelight source 1508, and/or the base station 108) are used to measure arelative location of the target 1504 to the base station 108 (e.g.,using the base station 108); generate a local map based on a pluralityof images acquired by the camera (e.g., of the wearable device 904),wherein the local map includes a relative location of the target 1504 tothe wearable device 904; and/or orient the local map to an environmentof the base station 108 based on the relative location of the target1504 to the base station 108, the relative location of the target 1504to the wearable device 904, and/or a relative location of the basestation 108 to the environment. A display (e.g., display 120 in FIG. 12)of the wearable device 904 can present a virtual object (e.g., virtualobject 1024) in relation to the environment based on orienting the localmap of the wearable device 904 with the environment. In someembodiments, the wearable device 904 is used to measure one or morecoordinates of a physical object based on orienting the local map of thewearable device 904 with the environment. For example, the wearabledevice 904 could be used to measure coordinates and/or orientation ofthe machine-readable code 1020 in FIG. 12.

By using the target 1504 as truth, virtual objects can be shown to auser, and/or measurements can be made, in relation to the environment.Though the target 1504 has been described as formed by light, othertargets could be used. For example, a feature in the environment couldbe used as a target. An example of a feature could be corner 1512. Thebase station 108 measures relative position of the feature (e.g., corner1512) in relation to the base station 108. The wearable device 904measures relative position of the wearable device 904 in relation to thefeature (e.g., corner 1512), and can orient the local map in relation tothe environment based on the relation of the wearable device 904 to thefeature and the relation of the feature to the base station 108.

In another example, the base station 108 itself can be used as a target.The wearable device 904 measures a relative position of the wearabledevice 904 in relation to the base station 108, and orients the localmap of the wearable device 904 to the environment based on the relationof the base station 108 to the wearable device 904. In some embodiments,the wearable device 904 has one or more cameras positioned to look to aside, a back, and/or above a user to track one or more targets 1504. Forexample, the wearable device 904 can have a camera positioned to lookbehind a user to image the base station 108 while the user is lookingaway from the base station 108, and/or the wearable device 904 can havea camera look “up” to view a target on a ceiling.

In some embodiments, a reflector is placed on the light source 1508and/or other locations in the environment. Reflector on the light source1508 can aid in determining a relative position of the light source 1508with respect to the base station 108. A reflector could be placed on astand (e.g., a tripod) and placed in a position within line of sight ofboth the base station 108 and the wearable device 904, wherein thereflector is used as a target for orienting the local map of thewearable device 904 with the environment. Multiple,uniquely-identifiable targets can be placed within an environment toprovide a user redundancy and/or availability of targets in anenvironment having several obstructions 1520. If a target (e.g., target1504) is not within a field of view of a camera of the wearable device904, then truth can be transferred to the wearable device 904 (e.g., asdiscussed in relation to FIGS. 11-13, wherein the reflector on thesurveying pole 916 can be considered a target).

In some embodiments, a target (e.g., target 1504) that is separate fromthe base station 108 is used because the base station 108 might not bein line of sight with the wearable device 904 (e.g., because ofobstruction 1520), and/or the base station 108 might be too far away fora camera of the wearable device 904 to adequately image the base station108. For example, the camera of the wearable device 904 has a wide fieldof view, which can make objects at a distance (e.g., physical objects ata distance equal to or greater than 3, 5, 10, 15, or more meters away)challenging to accurately position in relation to the wearable device904.

FIG. 16 is an embodiment of a movable design on a surface used as atarget 1504 for truth. In FIG. 16, an embodiment of a user perspectiveof an environment through an optical display 120 is shown. As theoptical display 120 is moved from right to left (e.g., a user turninghis head to look left), the target 1504 is moved from right to left(e.g., to stay within a field of view of the user). In some embodiments,eye-tracking data is obtained (e.g., using an eye-tracking capabilitiesof an augmented-reality system of the wearable device 904) to determinewhere a user is looking.

The eye-tracking data can be transmitted from the wearable device to thebase station and/or the light source. The light source moves the target1504 to stay within a field of view of the user based on theeye-tracking data. Thus one or more processors can be configured totrack an eye of the user, and move a position of the target 1504 basedon tracking the eye of the user.

In some embodiments, movement of the wearable device is tracked (e.g.,by image data and/or IMU data), and one or more processors areconfigured to move a position of the target 1504 in response to movementof the wearable device (e.g., translation or rotation of the wearabledevice could result in translation of the target 1504).

In some instances, the target 1504 cannot be moved to stay within afield of view of the user. In such instances, truth can be transferredfrom the target 1504 to the wearable device (e.g., similarly asdiscussed in the description of FIGS. 11-13).

The target 1504 is characterized by a design. The design can be assimple as a dot (e.g., a laser dot from a robotic total station). Thedesign can be a non-elliptical, two-dimensional design. For example, thedesign in FIG. 16 is an isosceles triangle oriented with a point of thetriangle directed vertically up (e.g., in a direction opposite of theforce of gravity). A galvanometer can be used to form the design (e.g.,by oscillating a mirror in a pattern to reflect a laser beam). Insteadof a laser, the design can be made by a mask covering an incoherentlight source.

In some embodiments, orientation data of the wearable device is at leastpartially based on the design of the target. For example, the targetcould be a vertical line, a horizontal line, or a plus sign “+” (e.g.,having a vertical line and a horizontal line) by translating a laserbeam vertically, horizontally, or both vertically and horizontally. Oneor more processors can be configured to ascertain an orientation of thewearable device in relation to the environment based on an orientationof a design of the target 1504. For example, in FIG. 16 an orientationof the target 1504 in relation to the environment is known, and thewearable device determines an orientation of the wearable device basedon the known orientation of the design of the target 1504. For example,vertical is determined by a point of the triangle and horizontal isdetermined by a horizontal line of the triangle.

In some configurations, the wearable device receives location data basedon the target 1504 and derives orientation data based on a feature inthe environment. For example, the target 1504 could be a laser dot. Thelaser dot does not provide relative orientation information of thewearable device to the environment. However, the wearable device couldidentify a horizontal edge 1604 and/or a vertical edge 1608 to determinerelative orientation of the wearable device to the environment, whereinthe horizontal edge 1604 and the vertical edge 1608 are features of theenvironment. A first horizontal edge 1604-1 is formed by an intersectionof a floor with a wall. A second horizontal edge 1604-2 is formed by anintersection of a ceiling with the wall. The vertical edge 1608 isformed by a corner of a wall of an entryway. Images of features acquiredby a camera of the wearable device provide information to the wearabledevice about orientation (e.g., with or without further data acquired byan IMU).

The optical display 120 produces a virtual object 1516 to appear inrelation to the environment. The virtual object 1516 in FIG. 16 is anoutline for an electrical box of a light switch positioned near theentryway. The virtual object 1516 can be positioned in relation tofeatures in the environment. For example, the virtual object 1516 isshown to the user so that a bottom of the virtual object 1516 is apredetermined distance (e.g., 48 inches; according to building code(s))from the first horizontal edge 1604-1 (e.g., from the floor).

Many variations using a target are possible. For example, differentpatterns for a design could be stored in a design library and chosen bythe user depending on a given use or preference. The user could selectthe design of the target 1504 using a menu presented to the user usingthe optical display 120 (e.g., an augmented-reality system presents amenu of five different target designs to the user, the user gestures toselect a specific target design, and the augmented-reality systemidentifies the gesture to select one of the five targets; theaugmented-reality system transmits the selection of the target the basestation; and the base station changes the pattern of the targetaccordingly). In some embodiments, the user can lock the target 1504 inplace so that the target 1504 does not move with the user (e.g., by theuser gesturing to lock the target). The user can also move the locationof the target 1504 (e.g., by gesture commands being identified by thewearable device and transmitted to the base station).

FIG. 17 illustrates a flowchart of an embodiment of a process 1700 forusing a target as truth. Process 1700 begins in step 1704 with directinglight, from a light source, to form a target on a surface. For example,light source 1508 directs an optical beam to form a target 1504 on awall in FIG. 15.

A relative location of the target is measured to a base station, step1708. For example, the base station 108 in FIG. 15 is a robotic totalstation, and the robotic total station measures a relative position ofthe target 1504 to the base station 108.

In step 1712, a local map is generated based on images acquired by acamera of a wearable device. The local map includes a relative locationof the target to the wearable device. For example, the wearable device904 in FIG. 15 is an augmented-reality device comprising a plurality ofcameras. The plurality of cameras acquire images, and the local map isgenerated based on images from the plurality of cameras.

The local map is oriented to the environment, in step 1716, based on therelative location of the target to the base station, the relativelocation of the target to the wearable device, and a relative locationof the base station to the environment. A virtual object is presented(e.g., to a user) on a display of the wearable device in relation to theenvironment, and/or one or more coordinates of a physical object ismeasured, based on orienting the local map of the wearable device withthe environment.

A method can comprise, and/or processors can be configured to performthe steps: ascertaining a depth and/or orientation of a physical objectin relation to the wearable device; calculating three-dimensionalcoordinates of the physical object in relation to the environment,wherein: the local map includes a relative location of the physicalobject to the wearable device, and/or calculating the three-dimensionalcoordinates of the physical object is based on the relative location ofthe physical object to the wearable device, the relative location of thephysical object to the wearable device, the relative location of thetarget to the base station, and/or the relative location of the basesstation to the environment; ascertaining an orientation of the wearabledevice in relation to the environment based on an orientation of adesign of the target; ascertaining an orientation of the wearable devicein relation to the environment based on a feature of the environment(e.g., an edge of a wall and the floor); tracking an eye of the user;moving a position of the target based on tracking the eye of the user;and/or moving a position of the target in response to movement of thewearable device.

In some embodiments, a wearable device is used to free up hands of auser (e.g., so the user can hold a surveying pole) or to increase afield of view of the user. In some embodiments, a device other than awearable device is used. For example, a mobile device (e.g., a tablet ora smart phone) having a camera and a screen could be used. The mobiledevice could be hand held or removably attached to the surveying pole.

FIGS. 18 and 19 demonstrate embodiments of orienting a local map of awearable device (e.g., wearable device 104 or 109) to an environment(e.g., to a coordinate system of a base station 108 for a layoutprocess). The wearable device 104 creates a local map of an area, but itcan be challenging to orient the local map to an (e.g., a room, aconstruction site, a building foundation, etc.).

FIG. 18 depicts an embodiment of a user marking a first position 1804.The user is using the wearable device 104 to control the base station108 for laying out points. The wearable device 104 creates a local mapof a scene (e.g., a 3D mesh using a SLAM algorithm and/or stereocameras). The base station 108 shines a laser spot at the first position1804. The base station 108 is oriented to the environment. The basestation 108 records the first position 1804 in relation to theenvironment and/or sends coordinates of the first position 1084 to thewearable device 104. For example, the base station 108 is a totalstation configured to precisely calculate a position of a laser spot.The user marks the first position 1804. In some embodiments the firstposition is marked with chalk, a pencil, a pen, a permanent marker, etc.(e.g., by the user drawing an ‘X’); is marked by using a piece of tape;is marked by the user driving in a nail or screw at the first position1804; is marked by a target (e.g., a sticker, a coin, a washer, a rubberwasher); or is marked by a laser spot from the base station 108. Theuser provides an instruction for the wearable device 104 to mark thefirst position 1804 in the local map. In some embodiments, the wearabledevice 104 orients the local map with the environment based on the firstposition 1804 and/or features identified in the local map (e.g.,comparing features in the local map to a CAD model of the environment).A path 1808 of a laser of the base station 108 is shown from the basestation 108 to the first position 1804.

FIG. 19 depicts embodiments of a user marking a second position 1904. Insome situations, the local map does not have many features. For example,the floor of a room could be concrete, which appears mostly the same tothe wearable device. Accordingly, in some embodiments, the secondposition 1904 is used to orient the local map to the environment. Insome embodiments, the second position 1904 is marked similarly as thefirst position 1804, though in some embodiments the second position 1904is marked differently than the first position 1804 (e.g., the firstposition 1804 marked with an X or blue token and the second position1904 marked with a triangle or a red token).

After marking the first position 1084 in the local map, the userinstructs the base station 108 to point to the second position 1904. Inthe embodiment shown, the base station 108 moves the laser (e.g., dropsor lifts) to point in a direction so that a line 1908 through the firstposition 1804 and the second position 1904 points toward the basestation 108; though in other embodiments the second position 1904 is toa side or other direction in relation to the first position 1804 (e.g.,not on line 1908. Having the first position 1804 and the second position1904 in the line 1908 toward the base station 108 can simplify someequations and/or indicate to the wearable device 104 a direction of thebase station 108 (e.g., so the wearable device 104 can track a positionof the base station 108). The path 1808 of the laser of the base station108 is shown from the base station 108 to the second position 1904. Thewearable device 104 has a field of view 1912. The first position 1804and the second position 1904 are within the field of view 1912 of thewearable device. In some embodiments, the base station 108 lifts thelaser, and the user moves away from the base station 108, so that thefirst position 1804 and the second position 1904 are between the userand the base station 108 (e.g., so that the user does not block the path1808 or the first position 1804). The local map is oriented to theenvironment based on both the first position 1804 and the secondposition 1904. For example, the wearable device 104 receives coordinatesof the first position 1804 and the second position 1904 from the basestation 108.

In some embodiments a method for orienting the local map to theenvironment comprises marking a first position (e.g., based on a laserfrom the base station 108 shining to the first position) in the localmap (e.g., the wearable device 104 identifying the first position inrelation to the local map); recording the first position in relation tothe environment (e.g., receiving coordinates of the first position fromthe base station 108). Marking a second position (e.g., based on thelaser from the base station 108 moving a laser spot to the secondposition) in the local map (e.g., the wearable device 104 identifyingthe first position in relation to the local map); recording the secondposition in relation to the environment (e.g., receiving coordinates ofthe second position from the base station 108); and/or orienting thelocal map to the environment based on the first position and the secondposition in both the local map and the environment.

FIG. 20 is a simplified block diagram of a computing device 2000.Computing device 2000 can implement some or all functions, behaviors,and/or capabilities described above that would use electronic storage orprocessing, as well as other functions, behaviors, or capabilities notexpressly described. Computing device 2000 includes a processingsubsystem 2002, a storage subsystem 2004, a user interface 2006, and/ora communication interface 2008. Computing device 2000 can also includeother components (not explicitly shown) such as a battery, powercontrollers, and other components operable to provide various enhancedcapabilities. In various embodiments, computing device 2000 can beimplemented in a desktop or laptop computer, mobile device (e.g., tabletcomputer, smart phone, mobile phone), wearable device, media device,application specific integrated circuits (ASICs), digital signalprocessors (DSPs), digital signal processing devices (DSPDs),programmable logic devices (PLDs), field programmable gate arrays(FPGAs), processors, controllers, micro-controllers, microprocessors, orelectronic units designed to perform a function or combination offunctions described above.

Storage subsystem 2004 can be implemented using a local storage and/orremovable storage medium, e.g., using disk, flash memory (e.g., securedigital card, universal serial bus flash drive), or any othernon-transitory storage medium, or a combination of media, and caninclude volatile and/or non-volatile storage media. Local storage caninclude random access memory (RAM), including dynamic RAM (DRAM), staticRAM (SRAM), or battery backed up RAM. In some embodiments, storagesubsystem 2004 can store one or more applications and/or operatingsystem programs to be executed by processing subsystem 2002, includingprograms to implement some or all operations described above that wouldbe performed using a computer. For example, storage subsystem 2004 canstore one or more code modules 2010 for implementing one or more methodsteps described above.

A firmware and/or software implementation may be implemented withmodules (e.g., procedures, functions, and so on). A machine-readablemedium tangibly embodying instructions may be used in implementingmethodologies described herein. Code modules 2010 (e.g., instructionsstored in memory) may be implemented within a processor or external tothe processor. As used herein, the term “memory” refers to a type oflong term, short term, volatile, nonvolatile, or other storage mediumand is not to be limited to any particular type of memory or number ofmemories or type of media upon which memory is stored.

Moreover, the term “storage medium” or “storage device” may representone or more memories for storing data, including read only memory (ROM),RAM, magnetic RAM, core memory, magnetic disk storage mediums, opticalstorage mediums, flash memory devices and/or other machine readablemediums for storing information. The term “machine-readable medium”includes, but is not limited to, portable or fixed storage devices,optical storage devices, wireless channels, and/or various other storagemediums capable of storing instruction(s) and/or data.

Furthermore, embodiments may be implemented by hardware, software,scripting languages, firmware, middleware, microcode, hardwaredescription languages, and/or any combination thereof. When implementedin software, firmware, middleware, scripting language, and/or microcode,program code or code segments to perform tasks may be stored in amachine readable medium such as a storage medium. A code segment (e.g.,code module 2010) or machine-executable instruction may represent aprocedure, a function, a subprogram, a program, a routine, a subroutine,a module, a software package, a script, a class, or a combination ofinstructions, data structures, and/or program statements. A code segmentmay be coupled to another code segment or a hardware circuit by passingand/or receiving information, data, arguments, parameters, and/or memorycontents. Information, arguments, parameters, data, etc. may be passed,forwarded, or transmitted by suitable means including memory sharing,message passing, token passing, network transmission, etc.

Implementation of the techniques, blocks, steps and means describedabove may be done in various ways. For example, these techniques,blocks, steps and means may be implemented in hardware, software, or acombination thereof. For a hardware implementation, the processing unitsmay be implemented within one or more ASICs, DSPs, DSPDs, PLDs, FPGAs,processors, controllers, micro-controllers, microprocessors, otherelectronic units designed to perform the functions described above,and/or a combination thereof.

Each code module 2010 may comprise sets of instructions (codes) embodiedon a computer-readable medium that directs a processor of a computingdevice 2000 to perform corresponding actions. The instructions may beconfigured to run in sequential order, in parallel (such as underdifferent processing threads), or in a combination thereof. Afterloading a code module 2010 on a general purpose computer system, thegeneral purpose computer is transformed into a special purpose computersystem.

Computer programs incorporating various features described herein (e.g.,in one or more code modules 2010) may be encoded and stored on variouscomputer readable storage media. Computer readable media encoded withthe program code may be packaged with a compatible electronic device, orthe program code may be provided separately from electronic devices(e.g., via Internet download or as a separately packagedcomputer-readable storage medium). Storage subsystem 2004 can also storeinformation useful for establishing network connections using thecommunication interface 2008.

User interface 2006 can include input devices (e.g., touch pad, touchscreen, scroll wheel, click wheel, dial, button, switch, keypad,microphone, etc.), as well as output devices (e.g., video screen,indicator lights, speakers, headphone jacks, virtual- oraugmented-reality display, etc.), together with supporting electronics(e.g., digital-to-analog or analog-to-digital converters, signalprocessors, etc.). A user can operate input devices of user interface2006 to invoke the functionality of computing device 2000 and can viewand/or hear output from computing device 2000 via output devices of userinterface 2006. For some embodiments, the user interface 2006 might notbe present (e.g., for a process using an ASIC).

Processing subsystem 2002 can be implemented as one or more processors(e.g., integrated circuits, one or more single-core or multi-coremicroprocessors, microcontrollers, central processing unit, graphicsprocessing unit, etc.). In operation, processing subsystem 2002 cancontrol the operation of computing device 2000. In some embodiments,processing subsystem 2002 can execute a variety of programs in responseto program code and can maintain multiple concurrently executingprograms or processes. At a given time, some or all of a program code tobe executed can reside in processing subsystem 2002 and/or in storagemedia, such as storage subsystem 2004. Through programming, processingsubsystem 2002 can provide various functionality for computing device2000. Processing subsystem 2002 can also execute other programs tocontrol other functions of computing device 2000, including programsthat may be stored in storage subsystem 2004.

Communication interface 2008 can provide voice and/or data communicationcapability for computing device 2000. In some embodiments, communicationinterface 2008 can include radio frequency (RF) transceiver componentsfor accessing wireless data networks (e.g., Wi-Fi network; 3G, 4G/LTE;etc.), mobile communication technologies, components for short-rangewireless communication (e.g., using Bluetooth communication standards,NFC, etc.), other components, or combinations of technologies. In someembodiments, communication interface 2008 can provide wired connectivity(e.g., universal serial bus, Ethernet, universal asynchronousreceiver/transmitter, etc.) in addition to, or in lieu of, a wirelessinterface. Communication interface 2008 can be implemented using acombination of hardware (e.g., driver circuits, antennas,modulators/demodulators, encoders/decoders, and other analog and/ordigital signal processing circuits) and software components. In someembodiments, communication interface 2008 can support multiplecommunication channels concurrently. In some embodiments thecommunication interface 2008 is not used.

It will be appreciated that computing device 2000 is illustrative andthat variations and modifications are possible. A computing device canhave various functionality not specifically described (e.g., voicecommunication via cellular telephone networks) and can includecomponents appropriate to such functionality.

Further, while the computing device 2000 is described with reference toparticular blocks, it is to be understood that these blocks are definedfor convenience of description and are not intended to imply aparticular physical arrangement of component parts. For example, theprocessing subsystem 2002, the storage subsystem, the user interface2006, and/or the communication interface 2008 can be in one device ordistributed among multiple devices.

Further, the blocks need not correspond to physically distinctcomponents. Blocks can be configured to perform various operations,e.g., by programming a processor or providing appropriate controlcircuitry, and various blocks might or might not be reconfigurabledepending on how an initial configuration is obtained. Embodiments ofthe present invention can be realized in a variety of apparatusincluding electronic devices implemented using a combination ofcircuitry and software. Electronic devices described herein can beimplemented using computing device 2000.

Various features described herein, e.g., methods, apparatus,computer-readable media and the like, can be realized using acombination of dedicated components, programmable processors, and/orother programmable devices. Processes described herein can beimplemented on the same processor or different processors. Wherecomponents are described as being configured to perform certainoperations, such configuration can be accomplished, e.g., by designingelectronic circuits to perform the operation, by programmingprogrammable electronic circuits (such as microprocessors) to performthe operation, or a combination thereof. Further, while the embodimentsdescribed above may make reference to specific hardware and softwarecomponents, those skilled in the art will appreciate that differentcombinations of hardware and/or software components may also be used andthat particular operations described as being implemented in hardwaremight be implemented in software or vice versa.

Specific details are given in the above description to provide anunderstanding of the embodiments. However, it is understood that theembodiments may be practiced without these specific details. In someinstances, well-known circuits, processes, algorithms, structures, andtechniques may be shown without unnecessary detail in order to avoidobscuring the embodiments.

While the principles of the disclosure have been described above inconnection with specific apparatus and methods, it is to be understoodthat this description is made only by way of example and not aslimitation on the scope of the disclosure. Embodiments were chosen anddescribed in order to explain the principles of the invention andpractical applications to enable others skilled in the art to utilizethe invention in various embodiments and with various modifications, asare suited to a particular use contemplated. It will be appreciated thatthe description is intended to cover modifications and equivalents.

Also, it is noted that the embodiments may be described as a processwhich is depicted as a flowchart, a flow diagram, a data flow diagram, astructure diagram, or a block diagram. Although a flowchart may describethe operations as a sequential process, many of the operations can beperformed in parallel or concurrently. In addition, the order of theoperations may be re-arranged. A process is terminated when itsoperations are completed, but could have additional steps not includedin the figure. A process may correspond to a method, a function, aprocedure, a subroutine, a subprogram, etc.

A recitation of “a”, “an”, or “the” is intended to mean “one or more”unless specifically indicated to the contrary. Patents, patentapplications, publications, and descriptions mentioned here areincorporated by reference in their entirety for all purposes. None isadmitted to be prior art.

What is claimed is:
 1. A system comprising: a light source configured todirect light to form a target on a surface; a wearable device separatefrom the light source, the wearable device comprising: a camera; and adisplay; one or more processors configured to: measure a relativelocation of the target to a base station; generate a local map of thewearable device based on a plurality of images acquired by the camera,wherein the plurality of images acquired by the camera include images ofthe target, and the local map includes a relative location of the targetto the wearable device; orient the local map to an environment of thebase station based on the relative location of the target to the basestation, the relative location of the target to the wearable device, anda relative location of the base station to the environment; and present,on the display of the wearable device, a virtual object in relation tothe environment and/or measure one or more coordinates of a physicalobject, based on orienting the local map of the wearable device with theenvironment.
 2. The system of claim 1, wherein the light source is alaser.
 3. The system of claim 1, wherein the target is a spot.
 4. Thesystem of claim 1, wherein the target is a non-elliptical,two-dimensional design.
 5. The system of claim 1, wherein the one ormore processors are further configured to ascertain an orientation ofthe wearable device in relation to the environment based on anorientation of a design of the target.
 6. The system of claim 1, whereinthe one or more processors are configured to receive a selection of adesign of the target from a user using the wearable device.
 7. Thesystem of claim 1, wherein the one or more processors are furtherconfigured to track an eye of a user, and move a position of the targetbased on tracking the eye of the user.
 8. The system of claim 1, whereinthe base station is separate from the light source.
 9. The system ofclaim 1, wherein the one or more processors are further configured tomove a position of the target in response to movement of the wearabledevice.
 10. A method comprising directing light, from a light source, toform a target on a surface; measuring a relative location of the targetto a base station; generating a local map of a wearable device based onimages acquired by a camera of the wearable device, wherein the imagesacquired by the camera include images of the target, and the local mapincludes a relative location of the target to the wearable device;orienting the local map to an environment of the base station based onthe relative location of the target to the base station, the relativelocation of the target to the wearable device, and a relative locationof the base station to the environment; and presenting, on a display ofthe wearable device, a virtual object in relation to the environmentand/or measuring one or more coordinates of a physical object, based onorienting the local map of the wearable device with the environment. 11.The method of claim 10, wherein the light source is a projector.
 12. Themethod of claim 10, wherein: the target is a spot; a location of thewearable device is based on relative position of the wearable device tothe spot; and orientation of the wearable device is based on features inthe environment imaged by the camera of the wearable device.
 13. Themethod of claim 10, further comprising ascertaining a depth and/ororientation of a physical object in relation to the wearable device. 14.The method of claim 10, further comprising calculating three-dimensionalcoordinates of the physical object in relation to the environment,wherein: the local map includes a relative location of the physicalobject to the wearable device; and calculating the three-dimensionalcoordinates of the physical object is based on the relative location ofthe physical object to the wearable device, the relative location of thetarget to the base station, and the relative location of the basesstation to the environment.
 15. The method of claim 10, furthercomprising ascertaining an orientation of the wearable device inrelation to the environment based on an orientation of a design of thetarget.
 16. The method of claim 10, further comprising: tracking an eyeof a user; and moving a position of the target based on tracking the eyeof the user.
 17. The method of claim 10, further comprising: receiving,from a user, a selection of a design; and directing light to form thetarget on the surface, wherein a shape of the target on the surfaceresembles the design.
 18. The method of claim 10, further comprisingmoving a position of the target in response to movement of the wearabledevice.
 19. A system comprising: a camera of a wearable device; adisplay of the wearable device; and one or more processors configuredto: measure a relative location of a target to a base station; generatea local map of the wearable device based on a plurality of imagesacquired by the camera, wherein the plurality of images acquired by thecamera include images of the target, and the local map includes arelative location of the target to the wearable device; orient the localmap to an environment of the base station based on the relative locationof the target to the base station, the relative location of the targetto the wearable device, and a relative location of the base station tothe environment; and present, on the display of the wearable device, avirtual object in relation to the environment and/or measure one or morecoordinates of a physical object, based on orienting the local map ofthe wearable device with the environment.
 20. The system of claim 19,wherein the one arc or more processors are configured to transmit anestimated location of the wearable device to the base station.